Creating Anomaly Detection System for Industrial Process with Deep Learning

This is an R Markdown Notebook

Your project - Detect Anomaly in Industrial Process with Deep Learning

Deep Learning… In this lecture I would like to esplore with you the DeepLearning we can do with H2O package in R…

Data Exploration

library(plotly)

Attaching package: 㤼㸱plotly㤼㸲

The following object is masked from 㤼㸱package:ggplot2㤼㸲:

    last_plot

The following object is masked from 㤼㸱package:stats㤼㸲:

    filter

The following object is masked from 㤼㸱package:graphics㤼㸲:

    layout

About the NORMAL dataset

NORM_2016 <- DF_NORMAL %>% select(2:11) %>% as.matrix() 
summary(NORM_2016)

This dataset was selected .. after .. the period of deep process maintenance

plot_ly(z = NORM_2016, type = "surface")

About the TEST dataset

This dataset contains some values that may indicate potential anomaly

TEST_2015 <- DF_TEST %>% select(2:11) %>% as.matrix()
plot_ly(z = TEST_2015, type = "surface")

About the ANOMALY dataset

This dataset contains some values that indicate the anomaly

TEST_2017 <- DF_ANOMALY %>% select(2:11) %>% as.matrix()
plot_ly(z = TEST_2017, type = "surface")

Summary: 3 dataset were selected!

Train Deep Learning Model

# ?h2o.deeplearning
normality_model <- h2o.deeplearning(x = names(train), 
                                     model_id = "DeepLearning_id20180317",
                                     training_frame = train, 
                                     activation = "Tanh", 
                                     autoencoder = TRUE, 
                                     hidden = c(8,5,8), 
                                     sparse = TRUE,
                                     l1 = 1e-4, 
                                     epochs = 100)

  |                                                                                                          
  |                                                                                                    |   0%
  |                                                                                                          
  |========================================                                                            |  40%
  |                                                                                                          
  |================================================================================                    |  80%
  |                                                                                                          
  |====================================================================================================| 100%

reconstruct dataset

Check MSE

# shutdown JVM
h2o.shutdown(prompt = F)

[1] TRUE

Homework:

• try different activation function, each time calculate MSE
• try sparse false parameter, what is changing?
• try high and very high complexity of the model, calculate MSE, conclude which one is the best
• try 10:100:10 is it work?
• try 200:200 will it work? what is the resulting MSE
• try 5 hidden layers
• try replicate_training data = False, how much time is different?
• weights_column this is the observation weight, try to add 1 column with observation weights of importance! see help

Ways to productionize the model

library(h2o)
# load the model
loaded_model <- h2o.loadModel("DeepLearning_id20180317")